Apparatus and method for fault-proof collection of imagery for underwater survey

ABSTRACT

An apparatus and method are presented comprising one or more sensors or cameras configured to rotate about a central motor. In some examples, the motor is configured to travel at a constant linear speed while the one or more cameras face downward and collect a set of images in a predetermined region of interest. The apparatus and method are configured for image acquisition with non-sequential image overlap. The apparatus and method are configured to eliminate gaps in image detection for fault-proof collection of imagery for an underwater survey. In some examples, long baseline (LBL) is utilized for mapping detected images to a location. In some examples, ultra-short baseline (USBL) is utilized for mapping detected images to a location. The apparatus and method are configured to utilize a simultaneous localization and mapping (SLAM) approach.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit under 35 U.S.C. § 119(e) to U.S.Provisional Application Ser. No. 62/752,070, titled “APPARATUS ANDMETHOD FOR FAULT-PROOF COLLECTION OF IMAGERY FOR UNDERWATER SURVEY,”filed on Oct. 29, 2018.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under National Oceanicand Atmospheric Administration (NOAA) grant number NA15NOS4000200. Thegovernment has certain rights in the invention.

TECHNICAL FIELD

The present invention relates to an apparatus and method for conductingunderwater surveys, collection and processing of image and/or videodata.

BACKGROUND

Underwater image and video surveys have become a common tool forresearchers and commercial companies involved in cable and pipelinelaying and inspection, building foundations for windfarms, and the like.

Researchers and experts have struggled to solve the SLAM problem inpractical settings, at least in part because SLAM processes require agreat deal of computational power to sense a sizable area and processthe resulting data to both map and localize. A completethree-dimensional SLAM solution may be highly computationally intensive,for example, requiring complex real-time particle filters, sub-mappingstrategies and/or hierarchical combination of metric topologicalrepresentations.

SUMMARY

Examples of the present disclosure meet one or more of the above needs,among others, by providing a system and method for collecting imagesand/or video, collecting position information associated with the imagesand/or video, and preparing of a map of an unknown environment using acomputer to analyze the collected information. The map may be atopological map. The map may capture the environment by combining imagesof pieces of the environment and connecting them in an appropriate way.The map may provide details with improved geometric accuracy whencompared to conventional systems and methods. The system and methoddisclosed herein may provide guaranteed high overlap between frames. Thesystem and method disclosed herein may provide fault-proof collection ofimagery. The system and method disclosed herein may be configured toprovide a 2-dimensional (2D) or 2.5-dimensional (2.5D) representation ofa predetermined region of interest.

Accordingly, pursuant to one aspect, there is provided an apparatus forimage collection, comprising an underwater vehicle, comprising a centralmotor comprising a rotating member, a set of arms couples to therotating member and extending away from the central motor, and one ormore cameras attached to a distal end of at least one arm of the set ofarms, wherein the underwater vehicle is configured to travel over apredetermined region and collect a set of images of the predeterminedregion.

Examples described herein may be further characterized by one or anycombination of features, such as a processor is configured to create afull mosaic image using non-sequential image overlap. In some examples,at least one of the central motor, a set of arms, and one or morecameras provided with rounded edges to facilitate hydrodynamic motion.In some examples, the set of arms is housed within a disc. In someexamples, the underwater vehicle is configured to utilize a longbaseline (LBL) approach to determine positioning for each imagedlocation in the predetermined region. In some examples, the underwatervehicle is configured to utilize an ultra-short baseline (USBL) approachto determine positioning for each imaged location in the predeterminedregion.

Pursuant to another aspect, an apparatus for collection of images isprovided. The apparatus comprises an underwater vehicle comprising amotor; and a set of sensors configured to rotate about the motor. Theunderwater vehicle may be configured to travel over a linear path in apredetermined region and detect information in the predetermined region.

Examples described herein may be further characterized by one or anycombination of features, such as the set of sensors being a set oftime-of-flight (TOF) sensors. In some examples, bathymetry informationis processed. In some examples, the set of sensors includes a set ofcameras. In some examples, the set of cameras are rotated at a constantrotational speed. In some examples, at least one of the set of sensorsand the motor are housed within a disc.

Pursuant to yet another aspect, a method is provided. The methodincludes acts of driving an underwater vehicle at a constant linearspeed, rotating one or more cameras about a central motor, capturing aset of images of a region to be mapped, and detecting position of theunderwater vehicle during image capture.

Examples described herein may be further characterized by one or anycombination of features, such as the one or more cameras are coupled toa processor configured for image co-registration using non-sequentialimage overlap. In some examples, one or more cameras that are rotated ata constant rotational speed, utilization of a long baseline (LBL)approach to detect position of the underwater vehicle in the region tobe mapped, and utilization of an ultra-short baseline (USBL) approach todetect position of the underwater vehicle in the region to be mapped. Insome examples, the one or more cameras are configured to send the set ofimages to a computer, to perform a bundle adjustment step, to performglobal optimization utilizing SLAM and non-sequential image overlap,and/or to compile the set of images into a map of the region.

Further aspects, advantages and areas of applicability will becomeapparent from the description provided herein. The description andspecific examples are intended for purposes of illustration only and arenot intended to limit the scope of the present disclosure.

DESCRIPTION OF THE DRAWINGS

The drawings described herein are for illustration purposes only and arenot intended to limit the scope of the present disclosure in any way.

FIG. 1 is an image mosaic constructed from images collected in alawnmower pattern, in one example of the disclosure.

FIG. 2 is an image survey pattern, in one example of the disclosure.

FIG. 3 is a top view of an example structure of the fault-proof imagecollection device, in one example of the disclosure.

FIG. 4 is a top view of an example structure of a hydrodynamic design ofan image collection device, in one example of the disclosure.

FIG. 5 is a top view of an example structure of a hydrodynamic design ofan image collection device, in one example of the disclosure.

FIG. 6 is a top view of an example path of the motor (straight line) andan example path of a pair of cameras (curved lines), in one example ofthe disclosure.

FIG. 7 is an intensity map showing coverage of the imaged swath, in oneexample of the disclosure.

FIG. 8 is a graph of relative sensitivity as a function of wavelengthfor an example time-of-flight (TOF) sensor.

FIG. 9 is a graph of area coverage for different linear speeds, inaccordance with one example of the disclosure.

FIG. 10 is an intensity map showing area coverage of the imaged swath,in accordance with one example of the disclosure.

FIG. 11 is a graph of area coverage for different snapshot periods, inaccordance with one example of the disclosure.

FIG. 12 is a flow diagram illustrating an example work flow, inaccordance with one example of the disclosure.

FIG. 13 is a perspective view of an example of an experimental setup fordemonstrating proof of concept, in accordance with one example of thedisclosure.

FIG. 14 is a top view of an example surface to be imaged provided with apseudorandom visual pattern, in accordance with one example of thedisclosure.

FIG. 15 is a top view of an example set of recovered camera positionsfollowing only sequential registration, in accordance with one exampleof the disclosure.

FIG. 16 is an example set of camera positions following correction afternon-sequential registration of frames and global optimization, inaccordance with one example of the disclosure.

FIG. 17 is a close up view of FIG. 14 further indicating marked frameposition numbers, in accordance with one example of the disclosure.

FIG. 18 is a graph of the difference between frame numbers and thenumber of the cross link illustrating the predictability of maximumoverlap for 25 non-sequential registrations, in accordance with oneexample of the disclosure.

FIG. 19 is an example mosaic following optimization using methods hereindescribed, in accordance with one example of the disclosure.

DETAILED DESCRIPTION

The following description provides one or more examples but is notintended to limit the present disclosure, application, or uses. Thepresent disclosure describes an apparatus that is configured to avoidtypical errors for surveys that follow standard protocols. Typicalsurveys may include image surveys, video surveys, and/or acousticsurveys.

Advantages of a video survey include fast coverage of large areas, highspatial resolution of the data (acoustic surveys, for example, yieldmuch lower resolution), ease of interpretation, and non-invasiveness, ascompared to sample collection. Sample collection, as used herein, mayrefer to acquisition of a single sample grab of sediment or severalsample grabs. Although sample collection is considered to provide aground truth, this method has a number of deficiencies. First, althougha sampler is dropped from a known position using GPS, the sampler is notguaranteed to hit the seafloor directly vertically below the knownposition due to currents, for instance. As a result of the imprecisepositioning, the sampler may not provide adequate information. Forexample, the sampler may hit the only boulder in the otherwise sandyarea. Second, areas of bedrock cannot be sampled at all. Third, samplecollection is an invasive technique that may damage the environment. Allof these issues are solved by utilizing the fault-proof collection ofimagery for underwater survey as described herein.

Video surveys may typically be performed by Remotely Operated Vehicles(ROV), Autonomous Underwater Vehicles (AUV), from towed platforms, orjust by divers with handheld cameras. The expected output is a detailedmap, much larger than a single camera footprint, often with heightinformation (2.5D), and preferably tied to a geographic location.

As will be seen, the devices and methods taught herein offer an improvedapparatus for capturing images using a Simultaneous Localization andMapping (SLAM) approach. SLAM requires finding non-consecutive framesshowing a previously-visited area of the seafloor. This process is knownas a loop-closure, and it allows for update of prior information about agiven trajectory. It is often a non-trivial task even for neighboringlines in the lawnmower pattern due to small overlap, slightly varyingcamera speed and altitude, and/or relatively featureless areas. In somesimulations there is no better solution than the brute forceapproach—trying to match each frame with each.

FIG. 1 illustrates a survey organized in a lawnmower (orboustrophodonic) pattern with high overlap between successive frames(for video >95%) and reasonable (˜>50%) overlap between neighboringruns. Image overlap is essential for construction of geometricallyundistorted maps. Stars represent regions of poor overlap in the imagereconstruction. The last processing step in the map construction is thebundle adjustment that allows for minimization of errors accumulating inthe process of consecutive adding of frames to the map. The wholeprocess of map building using this approach is often referred as SLAM.

FIG. 2 shows one example of supplementing the lawnmower pattern, whichmay be accomplished with a few crisscrossing lines or with a secondsimilar pattern rotated with respect to the first to guarantee thatcollected data can be processed using SLAM techniques.

Without accurate positioning of the underwater system, even carefullyplanned surveys sometimes leave gaps or areas of insufficientnon-consecutive overlap that introduce errors in resulting maps. Toavoid such errors, some surveyors lay ropes or cables on the seafloorprior to data collection which significantly lengthens the timenecessary for the survey. Even AUVs programmed to perform the surveymission can be thrown off course by a current not taken into account atthe planning stage.

As shown in FIG. 3, the apparatus 10 may include a central motor 50surrounded by one or more cameras 60 configured to rotate in a direction65 and capture image and/or video data of a predetermined region ofinterest. The footprint that may be accurately imaged may depend on avariety of factors. Factors to consider when determining an optimalconfiguration for image detection include camera speed, distance fromcamera 60 to imaged region, linear speed of the platform in a direction55, image resolution achievable with a given camera 60. Other factors toconsider may include if the camera has any tilt or if the imaged surfaceis rugged (i.e. not flat). For underwater applications, visibilityunderwater must also be considered; thus, for underwater applicationsthe camera 60 or set of cameras 60 must be positioned relatively closeto the seafloor. In some examples, the set of cameras 60 or set ofsensors may be positioned between about 2 m, about 5 m, about 10 m,about 20 m, or more above the ocean floor. In some examples, the set ofcameras 60 may be positioned between about 2 m and about 10 m above theocean floor.

One example uses an underwater motor 50 with an arm 70 attached by itscenter to the motor shaft such that the arm 70 rotates in a planeparallel to the seafloor in a direction 65. In some examples, one ormore cameras 60 are attached to the arms 70. One or more, two or more,three or more, five or more, or ten or more cameras 60 or sensors may beutilized for detection of a region of interest. In some examples, theone or more cameras 60 are positioned facing vertically down. In someexamples, the one or more cameras 60 are positioned at an angle withrespect to the seafloor. When the arms 70 rotate, the cameras 60 acquireimages of the circle with the diameter equal to the length of the arms70 and the size of the motor 50.

When the motor drives forward moving in a straight-line direction 55(linear motion, keeping constant altitude), the circle covered by therotating cameras 60 also shifts covering new areas (FIG. 6). The imagedarea depends on the size of the footprint of cameras 60, rotationalspeed of the arm 70, and linear speed of the apparatus 10. Withrelatively low linear speed, each area may be imaged at least twice—byboth cameras 60. With relatively high linear speed, the camera 60 whichis “currently ahead” of the motor may leave gaps which will be imaged bythe “currently behind” camera 60. With even higher linear speed, someareas of the seafloor may not be imaged at certain rotational speeds;however, this problem may be alleviated by utilizing a higher rotationalspeed of the arm 70. Note, that most of the imaged area is covered bythe forward facing and backward-facing sectors. However, the side areas(the furthest from the motor path) play an important role, as this iswhere the non-consecutive frames overlap occurs in a highly predictablemanner. Every rotation cycle provides four guaranteed overlap areas—onentry and on exit of the previous rotation. Thus, knowledge ofrotational speed and linear speed allows to foretell wherenon-sequential overlap occurs. Even without prior knowledge of thelinear speed it is sufficient to find non-sequential overlap only onceto predict the following occurrences.

Structurally, a set of cameras 60 is provided that are configured torotate in a direction 65 about a central motor 50. As the motor 50drives in a forward linear direction 55, the set of cameras 60 rotatesabout the central motor 50. FIG. 6 the trajectories taken by motor 50 ina linear direction 55 and the trajectories taken by cameras 60 in arotational direction 65. In some examples, conventional cameras are usedfor image collection such as, for example, digital cameras, underwatercameras, digital SLR's, and/or compact digital cameras. In otherexamples, time-of-flight (TOF) cameras may be implemented for imagecollection. In some examples, the one or more cameras 60 are configuredto look straight down during image collection. The set of cameras 60 maybe oriented so that each camera 60 individually is directed in adownward direction, enabling imaging of the seafloor. Positioning eachcamera 60 facing downward is configured to allow for the creation of aflat mosaic during image reconstruction. An arrangement of the cameras60 where the cameras 60 are downward facing may prevent or minimizeoblique imaging. Oblique imaging may introduce challenges as far asisolating structure from motion. During image reconstruction, shapes maybe reconstructed without having to isolate structure from motion ifoblique imaging is prevented or minimized.

In some examples, it may be desirable to produce 2.5D imagereconstructions using structure from motion. Structure from motionphotogrammetry may provide hyperscale landform models using imagesacquired from a range of digital cameras and optionally a network ofground control points. Structure from motion photogrammetry may providepoint cloud data. Structure from motion may be useful in remote orrugged environments where terrestrial laser scanning is limited byequipment portability and airborne laser scanning is limited by terrainroughness causing loss of data and image foreshortening. Structure frommotion may be applied in many settings such as rivers, badlands, sandycoastlines, fault zones, and coral reef settings. Structure from motionphotogrammetry may be configured to provide detailed surface topographyin unprecedented detail, multi-temporal data, elevation detection, ordetection of position and volumetric changes providing details on earthsurface processes, for example. Structure from motion may include usingdata acquired by a camera in free motion and performing postprocessingsuch that the resulting image shows a non-flat structure.

In some examples, the apparatus and methods disclosed herein can beutilized for measurement of underwater depth of lake or ocean floors.Cameras 60 may be configured for direct acquisition of bathymetry, orunderwater depth of the ocean floor, including locating peaks andvalleys and regions of flat terrain. For bathymetry or micro-bathymetryanalysis, the one or more cameras 60 may be replaced with one or moretime-of-flight (TOF) sensors, such as, for example a HamamatsuS11963-01CR sensor. This device may simultaneously acquire a 160×120array of distance measurements. The spectral response of the sensor(FIG. 8) may be configured for use with a green laser for illumination,whose light is only weakly absorbed by water.

The more than 100% coverage of the area of interest can be achieved forany reasonable camera footprint—its size requires only proper adjustmentof rotational and linear speeds. Structurally, the set of cameras 60 mayinclude 2, 3, 4, 5, 6, or more cameras. For a given set of cameras 60,if more cameras 60 are provided slower rotational speeds may berealizable. If fewer cameras 60 are provided, then a faster rotationalspeed would be required to capture images covering the same region.

FIGS. 6-7 are the result of simulations with certain parameters fixed(elevation, field of view, rotational speed, number of cameras). FIG. 6shows the linear path 55 of motor 50 and rotational trajectories 65 ofcameras 60 at the ends of the arms 70. FIG. 7 shows the correspondingcoverage of the imaged swath. Black areas (Q) are not imaged, dark gray(R) indicates less covered areas, and medium gray (S) indicates areasthat are more covered. Areas that are imaged most (up to 14 times forthis example) are shown in light gray (T).

FIG. 7 details overlap created in an example imaged swath. The imagedswath width is theoretically unlimited—it is determined by the armlength. However, in practice, the weight of the arm 70 and camerahousings and their drag may limit the swath width. To minimize the dragand to avoid turbulence, component housings may be provided with ahydrodynamic design. For example, the arms 70 connecting the cameras 60to the motor 50, the motor 50 itself, and the cameras 60 may be providedwith curved surfaces, may be formed in a disc shape, and/or may berounded to limit potential drag and/or turbulence. FIGS. 4-5 illustratealternate examples illustrating apparatus 110 and 120, respectively,showing some example hydrodynamic designs which provide curvature toarms 70.

The arm 70 with cameras 60 at the ends may be replaced with a solid diskthat may be heavier than the arm 70 but has a number of advantages.First, the solid disk has a reduced drag than the arm 70 while rotating.Second, the solid disk allows for mounting more than two cameras 60along its edge. The latter guarantees larger than 100% coverage withfaster linear speed.

The motor may be handheld as the rotating arm 70 has properties similarto a gyroscope and once it starts to rotate, it tends to stay in thesame plane of rotation. However, it may be mounted on a self-propellingplatform that helps to keep the motor in a vertical position with littlecorrections from a diver and helps to maintain almost constant linearspeed. The whole device may also be mounted on a vehicle, such as ROV orAUV.

One advantage of the proposed device is that, in some examples, a surveyexploiting this device can be successfully performed by non-trainedpersonnel. The proposed device eliminates the need to lay ropes or othermarkers on the seafloor. The proposed device eliminates the need todevise a mission plan for an AUV. When the surveyed area size exceedsthe swath covered in one transect and there is still a need to employthe lawnmower pattern, there are less tough restrictions on the accuracyof the path in the opposite direction. More specifically, the proposeddevice is configured to compensate by matching non-sequential images aswell as sequential images to form an accurate mosaic.

In the apparatus of the present disclosure, a relationship existsbetween linear speed of the motor, rotational speed of the arms 70extending from the motor, field of view of the one or more cameras 60,length of the arms 70, or diameter of the disc, altitude of the camera60 above the surface being imaged. For example, when the linear speed islow, rotational speed can also be low. However, for higher linearspeeds, the rotational speed would need to be higher or more cameras 60would need to be added to the system to cover the same area.

Coverage of the surveyed area using the apparatus and method describedherein may depend on parameters including positional elevation above thearea to be imaged, vertical field of view (FOV), rotational speed, andnumber of cameras. The vertical field of view of the camera FoV_(v)(degrees) may be in the range of 30 degrees to 50 degrees, 35 degrees to45 degrees, or 37 degrees to 42 degrees. The rotational speed v_(r) maybe in the range of 0.1 revolutions per second to 2 revolutions persecond, 0.7 revolutions per second to 1.7 revolutions per second, or 1.0revolutions per second to 1.5 revolutions per second. The number ofcameras N_(c) may be in the range of 1 camera to 6 cameras. Variationsin these ranges may also result in effective system performance.

Image or data acquisition would need to occur at a rate which is fastenough (i.e. a sufficiently short time period between sequentialsnapshots) to support a system configured to provide gap free image ordata acquisition. For a given time period between sequential snapshots,the maximum linear speed of the device can be estimated as:

V _(linear)≤2Hv _(r) N _(c) tan(FoV _(v)/2)

The above formula provides the upper limit of the linear speed thatguarantees that there will be no gaps in detected area. Substitutingtypical values of parameters, maximum linear speed of the motor may liein the range from 0.5 m/s to 112 m/s. Maximum linear speed for any givenexample may depend on hardware, water clarity, or other factors. Notethat maximum linear speed does not depend on the length of the arm, thedistal end of which supports and connects the one or more cameras.However, linear speed, which may be limited by potential drag,determines the width of the swath imaged by a single pass of the device.For a specific set of hardware and survey conditions, the maximum linearspeed may be determined by simulation, which may also provideinformation about overlapping non-sequential frames.

Taking into account the frequency of snapshots would yield a morecomplex version of the formula for determining maximum linear speed.

To provide some insight into the dependence of area coverage onimportant parameters such as rotational speed, linear speed, size ofarea to be imaged, and snapshot period, FIGS. 6-7 and FIGS. 9-11 provideexample results of simulations. A given image is assumed to have a 4:3aspect ratio, footprint height is equal to 0.6 of the arm length. Notethat footprint height is related to the altitude and the field of view:

h=2H tan(FoV _(v)/2)

FIGS. 9-11 are the result of simulations with most of the parameters(elevation, field of view, rotational speed, number of cameras) fixed.FIG. 9 illustrates that for higher relative linear speed, imaged areacoverage is relatively lower. For lower relative linear speed, imagedarea coverage is relatively higher. For a linear speed of 10 m/s, forexample, almost all covered area is being imaged between 5 to 15 times.For higher linear speeds of about 30 m/s, coverage becomes thinner. Forlinear speeds of about 32 m/s and higher, gaps start to appear in theimaged swath.

FIG. 10 shows central portions of the imaged swath highlighted withstars as well as jagged edges of the imaged swath, both of whichindicate gaps in detected regions of the imaged swath. FIG. 10,represents a simulation indicating the presence of gaps in the imagedarea at linear speeds of about 32.5 m/s.

FIG. 11 illustrates that for a longer relative snapshot period, theimaged area coverage is relatively lower. For a shorter relativesnapshot period, the imaged area coverage is relatively higher. Snapshotperiod is defined herein as the time between snapshots taken by acamera. Thus, shorter snapshot periods make coverage denser, whilelonger snapshot period make coverage thinner. Even longer snapshotperiods may lead to the appearance of gaps in the imaged swath.

FIG. 12 details an example work flow which can be performed by thedevice shown in FIG. 3, for example. In step 100, image data iscollected. In step 110, position data is recovered for each image.Camera positions are recovered based on the content of images withoverlapping footprints (non-sequential overlap). In step 120, frames aredetermined with non-sequential image overlap and these images areco-registered. In step 130, bundle adjustment is performed by aprocessor, utilizing both image data and position data. The bundleadjustment step allows for minimization of errors accumulating in theprocess of consecutive adding of frames to the map. Given a set ofimages depicting a number of 3D points from different viewpoints, bundleadjustment can be defined as the problem of simultaneously refining the3D coordinates describing the scene geometry, the parameters of therelative motion, and the optical characteristics of the one or morecameras employed to acquire a set of images. Bundle adjustment may beemployed as a last step in a feature-based reconstruction algorithm. Instep 140, a map is created using processed image data corresponding to aset of locations.

FIG. 13 illustrates an example of an experimental setup wherein a pairof cameras 60 are positioned one at each end of arms 70. Motor 50 isconnected to a moving platform 90 via vertically extending shaft 80.Arms 70 extend horizontally from shaft 80. Arms 70 are positionedroughly parallel to the surface to be imaged. In the experimental setupshown in FIG. 13, moving platform 90 is set up to move at a controlledlinear speed. The experimental setup shown in FIG. 13 is equivalent toan AUV or RUV moving at a controlled linear speed underwater. Cameras 60are set up to rotate at a rate of 6 rotations/minute. Linear motion ofthe moving platform 90 in combination with the rotation speed of 6rotations/minute resulted in non-sequential overlap which wassufficiently high to produce a near-guaranteed absence of gaps incoverage. Registration of only sequential frames resulted in therecovered camera positions shown in FIG. 15.

FIG. 14 illustrates an example surface to be imaged provided with apseudorandom visual pattern. A pseudorandom pattern of black and whiteregions is painted on mesh safety fencing. Detection of images of thefence containing the pseudorandom pattern allows for easier registrationof one image with another. FIG. 14 represents a typical frame acquiredby a camera shown in FIG. 13, for example. It should be noted thatdetection of such a surface underwater should account for the fact thatsome warping may occur in the mesh layer due to underwater fluctuations.

In FIG. 15, a top view of an example set of recovered camera positionsfollowing sequential registration only is depicted. Some frames aremarked for clarity. It can be seen that some camera tilt did exist as aresult of the converging camera positions over time.Non-perpendicularity of the optical axis of camera 60 andnon-perpendicularity of the imaged surface contribute to theseartifacts. Application of the SLAM technique described hereincompensates for the accumulating error in positioning. To accomplishthis, registration, or image matching, of non-sequential frames isnecessary. It was found that the maximum non-sequential overlap isachieved for pairs of frames separated by 31-32 frames. After theregistration of such pairs of frames and global optimization, camerapositions were corrected, as shown in FIG. 16.

FIG. 16 illustrates the set of camera positions from FIG. 15 followingcorrection after non-sequential registration of frames and globaloptimization. In FIG. 16, solid lines indicate links between sequentialframes (sequential link) and dotted lines indicate links betweennon-sequential frames (cross-link).

FIG. 17 illustrates a close up view of FIG. 16 further indicating markedframe position numbers, indicating that for the example shown here, itwas either at frame 31 or frame 32 that non-sequential overlap occurred.Numbers listed above the dotted line indicate frame numbers and numberslisted below the dotted line indicate the difference between framenumbers for a given non-sequential overlap. Maximum overlap is achieved,in this example, between frames with numbers separated by 31 or 32frames.

FIG. 18 illustrates that the predictability of overlap is very high.Maximum overlap for 25 non-sequential registrations was achieved betweenframe 31 or frame 32. This is consistent when looking either from leftto right or right to left (i.e. either from low frame numbers to highframe numbers or from high frame numbers to low frame numbers).

It is important to note that the system and methods described hereinprovide for high predictability of non-sequential overlap without theuse of manual markers placed in a region to be imaged. The high imageoverlap provided by the system described herein allows for predictableguaranteed non-sequential overlap for future captured frames once thefirst non-sequential overlap has been identified.

Following global optimization utilizing SLAM and non-sequential imageoverlap, a full mosaic from all frames can be built. In the proof ofconcept, about 750 frames were captured. FIG. 19 illustrates a portionof the full reconstructed mosaic resulting from the optimization usingthe system and methods herein described. This example mosaic wasreconstructed from approximately 600 frames.

Improved Image Overlap and Map Quality

Functionally, the apparatus described herein is configured to detectimages using a central motor 50, a set of arms 70 extending from thecentral motor, and one or more rotating cameras 60 or sensors fixed tothe set of arms 70. The apparatus 10 is configured to detect, or image,an area. The apparatus 10 is configured to provide improved imageoverlap of the predetermined region of interest as compared toconventional systems. The apparatus 10 is configured to minimize missingportions of the region to be mapped or analyzed. The apparatus 10described herein is configured to provide improved image overlap andthereby provide a more complete and higher quality map of apredetermined region of interest.

In some examples, super-resolution techniques may be employed duringimage acquisition. Super-resolution techniques may enhance theresolution of the imaging system. For example, in some examples, a givensensor may use both optics and ultrasound for detection of apredetermined region of interest. In some examples, optical superresolution may be employed to transcend the diffraction limit of thesystem. Optical super resolution may involve substitutingspatial-frequency bands, multiplexing spatial-frequency bands, use ofmultiple parameters within a traditional diffraction limit, and/orprobing of a near-field electromagnetic disturbance.

In some examples, geometrical super resolution may be employed toenhance the resolution of digital imaging sensors. Geometrical superresolution may involve multi-exposure image noise reduction, singleframe deblurring, and/or sub-pixel image localization. Multiple imagesof the same scene allow one to apply super resolution techniques thatresult in images with resolution higher than the original ones. In someexamples, radar and sonar imaging may be enhanced using substancedecomposition-based methods and compressed sensing-based algorithms.Super resolution techniques are configured generally to improveresolution of the resulting reconstructed map.

In some examples of the present disclosure, a computer is provided forprocessing received images and associated image locations. The computermay be configured to construct or update a map of an unknownenvironment.

Underwater Positioning Systems

Underwater, two approaches may be utilized which track underwatervehicles using acoustics to determine positioning. In the long baseline(LBL) approach several transponders are placed in an area of interest,or worksite, to be mapped. In some examples, a network of sea-floormounted transponders may be used as reference points for navigation.Transponders may be placed around the perimeter of a worksite. The LBLtechnique enables matching the location of each image with highaccuracy. Accuracy is generally better than 1 meter and can reach a fewcentimeters accuracy. Positional accuracy and position stability may beindependent of water depth.

In the long baseline approach, the position of an AUV or ROV can bedetermined by acoustically measuring the distance from the AUV or ROV tothree or more seafloor deployed baseline transponders. Rangemeasurements may be supplemented by depth data from pressure sensors onthe transponders. Range measurements may be used to triangulate thepositioning of the underwater vehicle.

In some examples, a vehicle mounted interrogator (A) may send a signal,which is received by a set of baseline transponders (B, C, D). Vehiclemounted interrogator (A) may send a signal at a given frequency X andreceive a signal at a given frequency Y1, Y2, and Y3, for example, fromeach of baseline transponders (B, C, D), respectively. A distance (d)and position of the vehicle from each baseline transponder may becalculated using time for each signal to reach the vehicle from eachbaseline transponder according to d=v*t, where velocity (v) is the speedof sound, approximately 1500 m/s. The speed of sound may vary dependingon temperature, salinity, and even sound frequency. Calculated positionsmay be relative to the location of the baseline transponders. Positionsmay be converted to a geo-referenced coordinate system such aslatitude/longitude or UTM (universal transverse Mercator) ifgeo-locations of the baseline stations are first established.

LBL may provide advantages over alternative positioning systems asalternative measurement systems, such as ultra-short baseline (USBL),use shorter baselines where range disturbances of a given amount canresult in much larger position errors.

In some examples, the USBL approach may be used to track image position.Ultra-short baseline is an alternative method of underwater acousticpositioning. In USBL, a transceiver may be mounted to a transceiver on apole under a ship and a transponder may be mounted on the seafloor or ona vehicle, such as an ROV or AUV. A computer may be used to calculate aposition from the ranges and bearings measured by the transceiver.

An acoustic pulse may be transmitted by the transceiver and detected bythe transponder, which then replies with an acoustic pulse of its own.The return pulse is detected by the transceiver aboard the ship. Thetime from the transmission of the initial acoustic pulse until the replyis detected is measured by the USBL system and is converted into arange.

To calculate subsea positioning, the USBL calculates both a range and anangle from the transceiver to the subsea transponder. The transceivermay contain an array of transducers for measuring received signal angle.A method called “phase-differencing” within the transducer array is usedto calculate the direction to the subsea transponder.

Either the long baseline or ultra-short baseline approach for acousticpositioning may be used to supplement the system and methods describedherein by providing position information for a given image. Positioninginformation from LBL is typically on the level of meters and from USBLis typically on the level of tens of centimeters. In particular, use ofLBL or USBL approaches may be helpful for in cases where the proposeddevice is used with a boustrophodonic pattern to provide a roughalignment.

However, it is important to note that creation of a mosaic using thesystem and methods described herein is achievable with pixel levelaccuracy. Creation of a mosaic with millimeter or even sub-millimeteraccuracy is achievable using the system and methods described herein.The system and methods described herein are configured to produce aseamless or near-seamless mosaic as a result of image processingtechniques including bundle adjustment, non-sequential image overlap,global optimization, and utilizing the SLAM approach.

What is claimed is:
 1. An apparatus for image collection, comprising: anunderwater vehicle, comprising a central motor comprising a rotatingmember; a set of arms coupled to the rotating member and extending awayfrom the central motor; and one or more cameras attached to a distal endof at least one arm of the set of arms, wherein the underwater vehicleis configured to travel over a predetermined region and collect a set ofimages of the predetermined region.
 2. The apparatus of claim 1, furthercomprising a processor configured to create a full mosaic image usingnon-sequential image overlap.
 3. The apparatus of claim 1, wherein atleast one of the central motor, the set of arms, and the one or morecameras is provided with rounded edges to facilitate hydrodynamicmotion.
 4. The apparatus of claim 3, wherein the set of arms is housedwithin a disc.
 5. The apparatus of claim 1, wherein the underwatervehicle is configured to utilize a long baseline (LBL) approach todetermine positioning for each imaged location in the predeterminedregion.
 6. The apparatus of claim 1, wherein the underwater vehicle isconfigured to utilize an ultra-short baseline (USBL) approach todetermine positioning for each imaged location in the predeterminedregion.
 7. An apparatus for collection of images, comprising: anunderwater vehicle comprising a motor; and a set of sensors configuredto rotate about the motor, wherein the underwater vehicle is configuredto travel over a linear path in a predetermined region and detectinformation in the predetermined region.
 8. The apparatus of claim 7,wherein the set of sensors are a set of time-of-flight (TOF) sensors. 9.The apparatus of claim 8, wherein the information is bathymetryinformation.
 10. The apparatus of claim 7, wherein the set of sensorsare a set of cameras.
 11. The apparatus of claim 10, wherein the set ofcameras are rotated at a constant rotational speed.
 12. The apparatus ofclaim 7, wherein at least one of the set of sensors and the motor arehoused within a disc.
 13. A method, comprising: driving an underwatervehicle at a constant linear speed; rotating one or more cameras about acentral motor; capturing a set of images of a region to be mapped; anddetecting position of the underwater vehicle during image capture. 14.The method of claim 13, wherein the one or more cameras are coupled to aprocessor configured for image co-registration using non-sequentialimage overlap.
 15. The method of claim 13, wherein the one or morecameras are rotated at a constant rotational speed.
 16. The method ofclaim 13, wherein a long baseline (LBL) approach is utilized to detectposition of the underwater vehicle in the region to be mapped.
 17. Themethod of claim 13, wherein an ultra-short baseline (USBL) approach isutilized to detect position of the underwater vehicle in the region tobe mapped.
 18. The method of claim 13, further comprising sending theset of images to a computer.
 19. The method of claim 18, furthercomprising performing a bundle adjustment step.
 20. The method of claim18, further comprising performing global optimization utilizing SLAM andnon-sequential image overlap.
 21. The method of claim 20, furthercomprising compiling the set of images into a map of the region.